Papers with Multilingual neural machine translation
Learning Language Specific Sub-network for Multilingual Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Multilingual neural machine translation models suffer from performance degradation when learning multiple languages. |
| Approach: | They propose to use LaSS to jointly train a single unified multilingual MT model. |
| Outcome: | The proposed model gains on 36 language pairs by up to 1.2 BLEU and zero-shot translation with 8.3 BLUE on 30 language pairs. |
Gradient-based Gradual Pruning for Language-Specific Multilingual Neural Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual neural machine translation suffers from performance degradation in high-resource languages compared to bilingual counterparts. |
| Approach: | They propose a gradient-based gradual pruning technique for multilingual neural machine translation that allows for partial parameter sharing across language pairs to alleviate interference. |
| Outcome: | The proposed approach yields a notable performance gain on IWSLT and WMT datasets. |
Multilingual Neural Machine Translation with Language Clustering (D19-1)
Copied to clipboard
| Challenge: | Existing work on multilingual neural machine translation has been neglected due to its burdensome training process. |
| Approach: | They develop a framework that clusters languages into different groups and trains one multilingual model for each cluster. |
| Outcome: | The proposed model reduces the cost of training and improves translation accuracy. |
Improving Zero-Shot Translation by Disentangling Positional Information (2021.acl-long)
Copied to clipboard
| Challenge: | Multilingual neural machine translation has shown the capability of directly translating between language pairs unseen in training, i.e. zero-shot translation. |
| Approach: | They propose to remove residual connections in an encoder layer to reduce the difficulty of generalizing to new translation directions. |
| Outcome: | The proposed model outperforms pivot-based translation in terms of quality and ease of integration of new languages. |
A Compact and Language-Sensitive Multilingual Translation Method (P19-1)
Copied to clipboard
| Challenge: | Existing paradigms for multilingual neural machine translation do not make full use of language commonality and parameter sharing. |
| Approach: | They propose a multilingual neural machine translation paradigm with one encoder-decoder model that makes full use of language commonality and parameter sharing. |
| Outcome: | The proposed method outperforms strong standard multilingual translation systems on WMT and IWSLT datasets. |
Language-aware Interlingua for Multilingual Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing multilingual neural machine translation models fail to capture diversity and specificity of different languages, resulting in inferior performance against individual models that are sufficiently trained. |
| Approach: | They propose to integrate a language-aware interlingua into an Encoder-Decoder architecture to learn a semantic representation from the semantic spaces of different languages while allowing for language-specific specialization of a particular language pair. |
| Outcome: | The proposed model achieves remarkable improvements over state-of-the-art multilingual NMT models and produces comparable performance with strong individual models. |
Towards Higher Pareto Frontier in Multilingual Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing Pareto optimization approaches are limited by the long-tailed distribution of multilingual corpora. |
| Approach: | They propose a Pareto mutual distillation framework that pushes the Paret frontier outwards rather than making trade-offs. |
| Outcome: | The proposed framework pushes the Pareto frontier outwards rather than making trade-offs, the authors show. |
Byte-based Multilingual NMT for Endangered Languages (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work has not studied how byte encoding can benefit endangered languages . multilingual neural machine translation (MNMT) models suffer from out-of-vocabulary issues and representation bottleneck . |
| Approach: | They propose a multilingual multilingual neural machine translation system to alleviate the representation bottleneck and improve translation performance in endangered languages. |
| Outcome: | The proposed system outperforms subword-based models on twelve languages up to +18.5 BLEU points, an 840% relative improvement over baseline models. |
Importance-based Neuron Allocation for Multilingual Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to multilingual neural machine translation tend to preserve general knowledge, but ignore language-specific knowledge. |
| Approach: | They propose to divide model neurons into general and language-specific parts based on their importance across languages. |
| Outcome: | The proposed model can preserve general knowledge but ignore language-specific knowledge on several languages, and is universal and cost-effective. |
Disentangling Pretrained Representation to Leverage Low-Resource Languages in Multilingual Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Multilingual neural machine translation requires an enormous dataset, leaving the low-resource language (LRL) underdeveloped. |
| Approach: | They evaluated five languages using a parallel corpus of 1,000 instances each and found a zero-shot improvement of 7.4 from the baseline score of 7.1 to a score of 15.5 at best. |
| Outcome: | The proposed model improves performance in the linguistically diverse country of Indonesia by 7.4 from baseline score of 7.1 to 15.5 at best. |
Addressing Asymmetry in Multilingual Neural Machine Translation with Fuzzy Task Clustering (2022.coling-1)
Copied to clipboard
| Challenge: | Existing clustering methods cannot handle asymmetric problem in multilingual NMT . existing models cannot handle the asymmetry problem since there are thousands of languages involved . |
| Approach: | They propose a fuzzy task clustering method to address the asymmetric problem in multilingual NMT by using task affinity as the clustering criterion. |
| Outcome: | The proposed method outperforms baselines for a multilingual model and the existing models. |
Distributionally Robust Multilingual Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual neural machine translation (MNMT) learns to translate multiple language pairs with a single model, but the data imbalance hinders it from performing uniformly across language pairs. |
| Approach: | They propose a distributionally robust optimization objective which minimizes the worst-case expected loss over the set of language pairs. |
| Outcome: | The proposed learning objective outperforms baseline methods on three sets of languages and shows that it is cost-effective and efficient. |
Adapting to Non-Centered Languages for Zero-shot Multilingual Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies attributed zero-shot translation to domination of central language, e.g. English, but we supplement this viewpoint with the strict dependence of non-centered languages. |
| Approach: | They propose a language-specific modeling method that adapts to non-centered languages to counteract the instability of zero-shot translation. |
| Outcome: | The proposed method performs better than baselines in centered data conditions and can easily fit non-centered data. |
Adaptive Token-level Cross-lingual Feature Mixing for Multilingual Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual neural machine translation models can translate multiple language pairs in a single model but lacks ability to capture language-specific features. |
| Approach: | They propose a token-level feature mixing method that captures different features and dynamically determines feature sharing across languages. |
| Outcome: | The proposed method outperforms baselines and can be extended to zero-shot translation. |
Gradient Consistency-based Parameter Allocation for Multilingual Neural Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Multilingual neural machine translation models are often prone to parameter interference . a common problem is that the model compromises with the language diversity to find a solution . |
| Approach: | They propose a method that allocates parameters based on consistency between the gradients of the individual language and the average gradient. |
| Outcome: | The proposed method reduces parameter interference and improves translation quality. |
Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing monolithic models for multilingual neural machine translation encounter parameter interference and inefficient inference for large models. |
| Approach: | They propose a detachable multi-way model that assigns each language to an individual branch . they use data from OPUS to build a translation benchmark covering 433 languages . |
| Outcome: | The proposed model outperforms existing models in OPUS and is faster than existing models. |
Mitigating Tokenization-Induced Distance Distortion in Long-Context Multilingual Machine Translation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing positional encodings rely on fixed token indices and implicitly assume uniform semantic density, which breaks down for long-context inputs. |
| Approach: | They propose a tokenization-aware adaptive positional encoding that conditions relative positional bias on input-level sequence length and fragmentation statistics. |
| Outcome: | The proposed model improves long-context robustness and accuracy over baselines. |